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(57) ABSTRACT 

A system for providing accurate, quantifiable and reproduc- 
ible assessments of an organization's performance based on 
predetermined criteria. The system includes a telecommu- 
nications infrastructure administered by a survey adminis- 
trator. Employees or non-employees of a company or other 
organization to be evaluated log on to the system and answer 
a plurality of questions relating to various aspects of the 
business entity's operations. The information may be 
obtained through Internet communications or through touch- 
tone telephone systems. The data is stored in a database and 
subsequently analyzed by the survey administrator for 
evaluation and forecasting of the business entity's perfor- 
mance. 
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1 . Do senior leaders guide the company in setting direction and sustaining 
effective leadership through the organization? 

O Yes 
• No 



Senior leaders fully guide and sustain effective leadership. 

O Almost Always Disagree 
% Rarely Agree (20%) 
O Seldom Agree (35%) 
O Agree (50%) 
O Mostly Agree (65%) 
O Strongly Agree (80%) 
O Almost Always Agree 
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Scores for a Sample Organization 


Evaluation Categories 


Total Points 


Percent of 
Total Points 


Points Earned 


1. Leadership 


110 


38.2 


42 . 


2. Strategic Planning 


80 


43.8 


35 


3. Customer and Market Focus 


80 


40 


32 


4. Information and Analysis 


80 


25 


20 


5. Human Resource Focus 


100 


44 


44 


6. Process ManaEement 


too 


36 


36 


7. Business Results 


450 


35.6 


160 


Totals 


1,000 


36.9 


369 
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ANONYMOUS RESPONDENT METHOD FOR 
EVALUATING BUSINESS PERFORMANCE 

BACKGROUND OF THE INVENTION 
[0001] 1. Field of the Invention 

[0002] The present invention relates generally to a survey 
system and apparatus for evaluating the results. More par- 
ticularly, it relates to a remotely accessible system for the 
collection of employee or non-employee survey responses to 
quantify various criteria relating to the operation of an 
organization. 

[0003] 2. Description of the Prior Art 

[0004] In an increasingly competitive marketplace, busi- 
ness entities strive to improve leadership, quality, customer 
satisfaction and other criteria that directly or indirectly relate 
to the ultimate profitability of the enterprise. Empirical 
evidence has established that business operations that excel 
in distinct and measurable attributes are far more likely to be 
profitable in commerce. As an example, the "Baldrige 
Index" is made up of publicly traded U.S. companies that 
have received the Malcolm Baldrige National Quality Award 
during the years 1988 to 1996. The Secretary of Commerce 
and the National Institute of Standards and Technology 
("NIST") were given the responsibility, under Public Law 
100-107, to develop and administer the Award with coop- 
eration and financial support from the private sector. NIST 
"invested" a hypothetical $1,000 in each of the six whole 
company winners of the Baldrige Award. The investments 
were tracked from the first business day of the month 
following the announcement of award recipients (or the date 
they began public trading) to Dec. 1, 1997. Adjustments 
were made for stock splits. Another $ 1,000 was hypotheti- 
cal^ invested in the Standard and Poor's Index 500 ("S&P") 
at the same time. NIST found that the group of six outper- 
formed the S&P 500 by more than 2.7 to 1, achieving a 394.5 
percent return on investment compared to a 146.9 percent 
return for the S&P 500. Therefore, it is desirable for an 
organization to administer periodic assessments of its opera- 
tions. 

[0005] The detailed assessment of a business operation is 
often a difficult, expensive, and time-consuming task. Typi- 
cally, upper level management may knowingly or inadvert- 
ently affect the accurate measure of information gathered for 
the assessment. Outside consultants are often employed to 
interview and observe the operation of the enterprise on-site. 
However, management may wish to influence the data for a 
number of reasons. Upper level management may be greatly 
affected by the results of the evaluation and may attempt to 
direct the outside consultant only to well performing opera- 
tions, shielding problem areas from discovery. In addition, 
the outside consultant may interview subordinate employees 
in the presence of their supervisor. This creates a poor 
environment for gaining candor from the subordinate 
employee on potential areas of improvement that are the 
responsibility of the supervisor. 

[0006] An evaluation by an outside consultant may require 
the consultant to travel to different geographical locations to 
assess a large operation with continuity. The consultant may 
also require the employee to stop productive work to allow 
time for the assessment interview. It is time-consuming for 
an independent consultant to conduct individual employee 



interviews, record the data, and assemble the information 
into a useful form. Furthermore, should a business wish to 
conduct periodic evaluations, there is no guarantee that the 
same consultant will be available. Therefore, the company 
cannot be assured that the next independent assessment 
performed will have the same consistency. Nor is there any 
assurance that secondary evaluations will produce meaning- 
ful results comparable to previous exercises. 

[0007] Should the company attempt to apply known eco- 
nomic principles to an "in-house" self-assessment, there is 
the potential that more harm than good may come from the 
endeavor. Many organizations begin the process of self- 
assessment with a shallow understanding of the performance 
criteria sought or the optimal method in which to gather the 
information. This self -assessment suffers from an insepa- 
rable relationship between the company's own evaluator and 
the upper management that may exert influence, not only on 
the examinees, but also on the examiner. 

[0008] Previous attempts have been made to provide busi- 
ness-related surveys such as described in U.S. Pat. No. 
5,551,880 to Bonnstetter et al. (the J 880 patent) which is 
incorporated herein by reference. The '880 patent describes 
a system for predicting the potential success of an individual 
for a particular job or task. A survey is conducted wherein 
the employee or potential employee submits information on 
behavioral and value preferences. The information is then 
analyzed and compared against standards for behavior and 
values previously resolved for specific employment. In a 
preferred embodiment of the *880 patent, the behavior and 
value questions are administered through software and 
evaluated via a predetermined algorithm. However, the 
system analyzes the individual employee and not the busi- 
ness entity as a whole. Nor does the J 880 patent describe an 
independent party to administer the survey to ensure confi- 
dentially, honesty, and a full disclosure of the employee's 
perceptions of the business entity. 

[0009] Therefore, there is a need in the art for a method 
and apparatus to provide a reliable self-assessment survey 
process that can be easily administered and scored with the 
accuracy and completeness of a well developed written 
narrative self-assessment. 

[0010] There is a further need in the art to integrate 
existing telecommunication technologies to avoid the time 
and labor intensive ordeal of the paper and pencil process or 
other traditional means of survey administration. This would 
allow the database of both individual responses and cumu- 
lative data to be completely external to the organization 
being assessed. 

[0011] There is a further need in the art to provide confi- 
dentiality to organizations and their employees participating 
in the survey. This provides a distinct opportunity to bench- 
mark the data to industry peers and to provide the objective 
assurance that all data is accurate and verifiable by a reliable 
survey administrator organization. 

[0012] However, in view of the prior art in at the time the 
present invention was made, it was not obvious to those of 
ordinary skill in the pertinent art how the identified needs 
could be fulfilled. 

SUMMARY OF THE INVENTION 
[0013] The present invention solves significant problems 
in the art by providing a telecommunications infrastructure 
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that supports the efficient, confidential and accurate measure 
of performance criteria relating to the performance of ao 
organization. A predetermined set of performance criteria 
are measured by the use of an automated employee and 
non-employee interview system whereby recorded 
responses are stored in a database which is utilized for a 
quantitative evaluation of various aspects of a business 
enterprise. 

[0014] The present invention comprises an array of survey 
questions for evaluating a plurality of predetermined criteria 
relating to the performance of an organization. A survey 
administrator controls a data-gathering interface means for 
employees or non-employees of the business to confiden- 
tially record answers to the survey questions. For each 
individual employee survey, the data-gathering interface 
records the employee's identification, their job classification 
and a business identifier. Pertinent data is also collected for 
each non-employee participating in the survey. The data- 
gathering interface may comprise an Internet web server, a 
touch-tone telephone entry system, or a paper and pencil 
system that compiles the answers by scanning them auto- 
matically. The survey administrator maintains a database of 
the responses to the survey questions and has the ability to 
archive past response data. The survey administrator then 
applies a selected formula to the survey data to evaluate the 
performance of the organization and produces a quantifiable 
result based on a plurality of business categories. 

[0015] In a preferred embodiment, an organization wish- 
ing to evaluate the performance of various key aspects of its 
operations selects one or more business performance mod- 
els. Each business performance model is measured using 
predetermined questions for employees of the business. A 
survey administrator performs the administration of the 
predetermined questions so that confidentially in the inter- 
view process insures candid answers to the questions. In one 
embodiment, responses to the questions are gathered utiliz- 
ing an Internet web server and are stored either locally on the 
server or at a remote location for the survey administrator to 
later compile and analyze. The web server provides the 
opportunity to quickly and efficiently gather information, 
particularly where various offices of an organization are 
separated by substantial geographic distance. The web 
server also may operate 24 hours per day at relatively low 
cost and permit a plurality of employees to enter in 
responses at the same time. 

[0016] In an alternative embodiment, a telephone system 
utilizing dual-tone multi-frequency ("DTMF") input pro- 
vides a means for employees to confidentially record infor- 
mation relating to the performance of a company. The 
DTMF entry has the advantage of not requiring the 
employee to have a machine capable of accessing the web 
server over the Internet. Using DTMF entry, the employee 
logs on using a code provided by the survey administrator. 
Once within the telephone system, pre-recorded questions 
are audibly played to the employee. At the conclusion of 
each question, the employee is prompted to record their 
response by depressing numerals on the telephone. 

[0017] In another embodiment, non-employees participate 
in the survey as well. However, the questions propounded to 
non-employees may differ from the questions propounded to 
employees. 

[0018] The data is readily compiled into a useable data- 
base format. At the conclusion of the data gathering stage, 



the survey administrator can easily analyze the recorded 
information and produce a report to the business entity 
evaluating the entity based on the pre -selected business 
performance criteria desired. Furthermore, the operation of 
the invention is highly reproducible and consistent. A base- 
line assessment can be repeated over time, allowing the 
organization to track quantifiable results and improvements. 

[0019] It should be noted that the current invention has a 
wide range of applications which may include heath care 
accreditation, educational institution assessment, and similar 
tasks. For example, the current invention may be applied to 
a Performance-based Organizational Effectiveness & Effi- 
ciency Tracking Program ("POET") wherein the organiza- 
tional effectiveness and efficiency of child welfare organi- 
zations are measured in addition to performance Indicators 
essential to success in a managed care environment. 

[0020] An advantage of the invention is that a complete 
assessment of a large corporation may be completed more 
rapidly and efficiently than using independent consultants 
that interview onsite. The reduced costs and speed permit a 
business to conduct more frequent performance evaluations 
that can be utilized to operate the business more profitably. 

[0021] Another advantage of the invention is that employ- 
ees of the business are provided with the opportunity to 
submit honest and candid answers to potentially sensitive 
questions. Because the opportunity for external influence to 
affect the employee's response is curtailed, the final evalu- 
ation of the business entity is more likely to uncover 
problem areas in its operations that may be subsequently 
improved. 

[0022] Another advantage of the invention is that the 
infrastructure of the evaluation is highly consistent over 
time. This permits the business entity to periodically mea- 
sure itself against previous assessments. 

[0023] These and other important objects, advantages, and 
features of the invention will become clear as this descrip- 
tion proceeds. 

[0024] The invention accordingly comprises the features 
of construction, combination of elements, and arrangement 
of parts that will be exemplified in the description set forth 
hereinafter and the scope of the invention will be indicated 
in the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0025] For a fuller understanding of the nature and objects 
of the invention, reference should be made to the following 
detailed description, taken in connection with the accompa- 
nying drawings; in which: 

[0026] FIG. 1 is a diagrammatical view of the survey 
process according to the invention; 

[0027] FIG. 2 is a diagrammatical view of an Internet- 
based method of retrieving and recording employee 
responses; 

[0028] FIG. 3 is an illustrative example of a survey 
question presented to a user from an Internet web server; 

[0029] FIG. 4 is a diagrammatical view of a telephone - 
based method of retrieving and recording employee 
responses; and 
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[0030] FIG. 5 is an illustrative example of an organiza- 
tion's performance scorecard based on the Malcolm Bald- 
rige criteria. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

[0031] Referring initially to FIG. 1, it will there be seen 
that the organization subject to the survey is denoted by the 
reference number 10 as a whole. Virtually all business 
entities have the following job classifications: executive staff 
30, management staff 40, support staff 50, and customer 
contact staff 60 as integral employee components which 
support most all business functions of the enterprise. It is 
each of these job classifications of employees who form the 
internal pool of respondents of the business entities. The 
login procedures for these internal respondents will be 
discussed in more detail later. 

[0032] In another embodiment of the invention, an exter- 
nal pool of respondents is also included. The external pool 
of respondents may include consumers, customers or clients 
of the business entity, i.e., those who are not employees or 
otherwise a part of the internal group of respondents. The 
external pool of respondents may also be adjusted as desired 
to include members of the board of directors or trustees, 
suppliers or vendors, people who perform outsourced opera- 
tions, competitors, partners, consultants, or any other rel- 
evant external respondent entity. The login procedure for 
these external respondents is similar to the login procedure 
for internal respondents, except that said login procedure 
identifies all members of the external pool of respondents as 
such and by their specific category such as customer, vendor, 
or the like. External respondents may also be queried as to 
their age, frequency of contacts with the business entity, and 
other information deemed pertinent. 

[0033] In many organizations there are one or more con- 
sultants or holders of other job titles who are on the business 
premises daily and whom might seem to be employees of the 
organization even though they are technically independent 
contractors. There are other jobs as well where the distinc- 
tion between employee or non-employee may not be easily 
ascertainable. This invention does not depend upon any legal 
analysis of who should be considered internal or external to 
the organization as a matter of legal definition because in 
some cases a non -employee such as an outside consultant 
might best be included as a member of the internal pool of 
respondents for survey purposes. Therefore, for convenience 
purposes, and not in any strict legal sense, the members of 
the pool of internal respondents are herein deemed employ- 
ees of the organization, and the members of the pool of 
external respondents are herein deemed non-employees of 
the organization. A survey administrator is free to determine 
the respective members of the pools of internal and external 
respondents in any way that best serves the purposes of the 
survey without regard to strict legal determinations of the 
legal status of the individual members of either pool. 

[0034] The system obtains information from individual 
employees or other respondents regarding their perception 
of performance criteria existing in the workplace. This is 
accomplished by utilizing a performance criteria survey. 
This survey is administered and controlled by a survey 
administrator denoted 20. The survey data is obtained by a 
data-gathering interface 70. The data -gathering interface 



may comprise a variety of telecommunication mediums 
which may include touch-tone telephone systems, hypertext 
language format ("HTML") Internet web servers, client- 
server wide area networks ("WANs") or local area networks 
("LANs"). Data-gathering interface 70 acts as a conduit to 
receive survey data 90 in response to plurality of predeter- 
mined survey questions. Each question contains several 
important aspects of a business function linked to vital 
performance abilities of the organization. 

[0035] The results of the survey are stored in a database 
100 for a baseline analysis of performance criteria and may 
also be used longitudinally in later intervals of time. Prior 
survey data from prior years 110 and 120 may be archived 
by the survey administrator 20 and later utilized as a 
base-line reference to the current evaluation of the organi- 
zation. Once a requisite number of respondents or all respon- 
dents complete surveys, an analysis 140 is performed on the 
survey data 90 by the survey administrator 20 and then 
communicated back to the organization 10. 

[0036] FIG. 2 provides an overview of an Internet-based 
data and tracking system. A remote user terminal 160 
accesses an Internet web server 70 through a public switched 
network ("PSN") 190. The survey administrator 20 provides 
the organization 10 with a series of codes for the login 210 
and test administration to begin. Typically, the organization 
10 will choose an alphanumeric password of ten digits or 
less for identification of the organization 10 as a whole. The 
survey administrator 20 records said password and opens it 
for usage in protected area of the website. Next the survey 
administrator 20 assigns a chronological range of numbers 
sufficient to match the total number of respondent employ- 
ees or non-employees elected and eligible to sit for the 
survey. To begin the login 210 process, the respondent 
accesses the world wide web and web server 70 using its 
uniform resource locator ("URL"). The URL may be pro- 
vided as a numerical-based Internet protocol ("IP") address 
or may be accessed by a registered domain name via a 
domain name server ("DNS"). 

[0037] It should also be noted that alternatives to web- 
based Internet communications may be employed. The alter- 
natives may include 3270 emulation, VT-100, ASCII or 
separate client-server applications. 

[0038] In a preferred embodiment, the first web page that 
appears offers an overview of the survey system, including 
its methodology of data collection, processing and aggrega- 
tion. This page also offers the user a link to proceed directly 
to the test. When this option is chosen a new screen appears 
which displays test selection criteria with a sample Likert 
scale. At the bottom of this screen, the user is prompted to 
complete three empty fields for password protection entry. 
The first field is for the business ID 220 password that 
represents a unique identification of the organization. The 
second field is for job classification entry 230 that may 
include coding for employee classifications of executive 
staff 30, management staff 40, support staff 50 or direct 
customer contact staff 60. It may also include coding for 
external respondents, mentioned above, such as customers, 
suppliers, and the like, and there may be a different code, if 
desired, for each category of external respondents. The third 
field is for a unique employee ID 240 that relates to the 
preset numbers matching the number of employees as indi- 
cated above. 
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[0039] Once this information is entered a confirmation 
process 250 is triggered that compares this keyed-in infor- 
mation to the information on server 70. If a match is detected 
the server then determines as at 260 if the user has previ- 
ously initiated the survey. If the user has previously initiated 
the survey, a cache 280 record of the user's answers is 
accessed and the user is only presented with the survey 
questions that were not answered. 

[0040] If a mismatch 170 in passwords is detected during 
the login process 210, an error screen is displayed and the 
user has the ability to reenter the three identification codes 
220, 230, and 240. It should be noted that additional fields 
may be added to the login process which may include length 
of employment, demographics and other data. Alternatively, 
a single login ID may be utilized which individually iden- 
tifies the employee's organization, job classification and 
personal identification. 

[0041] Upon successful login 210 and confirmation 250, 
the survey is commenced as at 270 with a series of questions 
presented in HTML format. Under each question is a form 
input section coded in HTML which may comprise a check 
box, radio button, or edit box. Alternatively, other input 
means compatible with the user's web browser may be 
utilized including ActiveX or Java. Responses to the input 
means are stored in a database 100 that may reside on the 
web server or at a remote location. The questions and answer 
input forms may be configured on a single scrollable page or 
on individual HTML pages joined by hyperlinks. 

[0042] FIG. 3 discloses a sample question screen as 
viewed through an Internet web browser comprising a 
plurality of input sections to each survey question topic. 
Users are prompted to respond to each of the questions in 
one or more components. In the following example, there 
are three components. Component A queries whether the 
respondent believes a specific performance criteria exists in 
their workplace. Component B queries the respondent to 
weigh their level of agreement or disagreement to a func- 
tional extrapolation of component A. Component C asks the 
respondent to check one of many methods how such a 
performance criteria could be improved. 

[0043] In the example of FIG. 3 two question components 
are presented. Question 1 proposes a query on the perceived 
leadership skills in the company and provides for a Boolean 
response. The first query is followed up by another query 
that requests a quantified response with seven possible 
entries. In the preferred operation of the Internet system, 
none of the responses are selected when the user first 
downloads the page to their web browser. Each response 
must be completed for the "next question button" to become 
available for next question cueing and for the responses of 
that user to be filed in the database 100. Should the user 
attempt to skip over the questions without responding, the 
internal controls validate whether a response has been 
entered and repeat the same page to the user until a com- 
pleted response is received by the web server. In a preferred 
embodiment, JavaScript is utilized to validate completed 
responses. The question-to-question responses are tempo- 
rarily stored in a cache 250 until all survey questions are 
completed and the data is then committed to database 100 
for analysis. An interrupt feature is available to all respon- 
dents who are unable to finish the survey in one sitting. If the 
respondent stops and logs off, he or she may restart the 



survey at any later time, by simply re-entering the password, 
job classification code and test ID. This procedure opens an 
incomplete survey and restores it to the last position or next 
available question from data stored in cache 250. When all 
questions are completed and a "save last answer question" 
button is selected, the survey is flagged as done 260 for final 
database 100 entry. Analysis 140 is available when all users 
complete their survey. Data from this part of the survey 
helps the organization evaluate the current level of organi- 
zational consistency about perceptions of effectiveness and 
efficiency between and among different levels of staff, across 
all operations, between staff members, and between and 
among the pool of external respondents as well, if any. As 
improvements are made the organization can re-visit the 
assessment in future years to track scores in key areas, to 
monitor improvements and to encourage greater perfor- 
mance. A score card system also provides a context for an 
organization's scores to be compared to the respective 
scores of other organizations. 

[0044] In the touch-tone telephone embodiment shown in 
FIG. 4, the questions are audibly prerecorded for playback. 
The employee or non-employee is given a telephone number 
to call with a set of access codes which may comprise a 
business ID, job classification and employee ID, if appli- 
cable, as described in connection with FIG, 2. The user dials 
into the system from a standard touch-tone phone 300. The 
user is prompted for each identifier which is entered by 
pressing the numbers of the touch-tone telephone. When a 
key pressed on a touch-tone telephone, a DTMF signal is 
emitted which is correlated to the numeral on the keypad by 
a system processor. These systems are widely used in 
automatic call distributor ("ACD") systems utilized by call 
centers and by banks and financial, institutions to provide 
automated account information. In the example, after the 
user is authenticated prerecorded segments 320 of the survey 
are played back to the user. As an option, the user may press 
the STAR (*) or POUND (#) key on their telephone to repeat 
the last playback segment. After a survey question is audibly 
presented to the user, the user then may answer the question 
depending on the response options using a DTMF input 330. 
For a Boolean question which requires a "yes" or "no" 
response, the user may be prompted to press "1" for "yes" 
or "2" for "no." Alternatively, should the survey question 
require a quantitative response, the user may be prompted to 
enter a range of numbers on the telephone keypad. For 
example, should the user "Almost Always Disagree," they 
would depress "1." If the user "Rarely Agrees," they would 
depress "2." The scale may continue so that should the user 
"Almost Always Agrees," they would depress "7." However, 
it should be noted that the telephonic entry is not limited to 
single digits. For example, should the user be required to 
indicate a percentage numeral of 55%, the entry may be 
executed by depressing in sequence: "5 ""5," and then the 
"#" key. Should the user fail to respond to the system with 
a DTMF signal 330 within a predetermined period of time, 
the system may replay as at 340 the previous question 
proposed. 

[0045] Both the Internet-based and telepbonically-based 
system preferably time-stamp the entries wherein each time 
a response is posted to the web server or entered into the 
telephone system, a corresponding record of the date and 
time is also stored in the database. This may be utilized for 
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an assessment of the amount of time needed on average to 
complete the evaluation and when each employee submitted 
their responses to the survey. 

[0046] FIG. 5 illustrates a diagrammatical overview of an 
organization's total quality output scorecard based on the 
Malcolm Baldrige criteria. However, other performance 
measurements and formulas may be applied within the scope 
of this invention. The purposes of the Baldrige Assessment 
are to promote an understanding of the requisites for per- 
formance achievement and competitiveness for business 
entities and to promote the sharing of information on suc- 
cessful business practices. The Baldrige Assessment consists 
of seven criteria: (1) leadership, (2) strategic planning, (3) 
customer and market focus, (4) information and analysis, (5) 
human resource development and management, (6) process 
management, and (7) business results. 

[0047] Leadership is evaluated by the ability of senior 
executives to guide the company and how the company 
addresses it responsibilities. Strategic planning examines 
how the company sets strategic directions and how it deter- 
mines key action plans. Customer and market focus exam- 
ines how the company determines the requirements and 
expectations of customers and markets. Information analysis 
examines the management, effective use, and analysis of 
data and information to support key company processes and 
the company's performance management system. Human 
resource development and management examines how the 
company enables its workforce to develop its full potential 
and how the workforce is aligned with the company* objec- 
tives. Process management examines aspects of how key 
production, delivery and support processes are designed, 
managed, and improved. Business results are examined by 
the company's performance and improvement in its key 
business areas: customer satisfaction, financial and market- 
place performance, human resources, supplier and partner 
performance, and operation performance. Business results 
are also evaluated by the performance of the company 
relative to its competitors. 

[0048] In a preferred embodiment, each question has both 
a yes/no and weighted score component so that data can be 
accurately tallied with sophisticated analysis and report 
generation. Instead of using a simple 1-5 Likert scale which 
is subject to a wide range of interpretation, the current 
invention uses quantified behavioral descriptors, dramati- 
cally increasing score reliability. As previously illustrated in 
FIG. 3 a written question is followed by a quantified 
(percentage-tagged) behavioral description. This form of 
test construction yields an increase in the consistency and 
accuracy of ratings as well as greater coherence of linkages 
between business processes. The method of utilizing a 
written question followed by a quantified description is well 
known in the health care industry. In health care it was 
discovered that by obtaining input this way from a cross 
section of functions and levels throughout the organization, 
a performance profile can be developed that not only iden- 
tifies strengths and areas for improvement, but deployment 
gaps as well. This is something that written narrative assess- 
ments alone do not effectively provide. Accurate survey 
data, based on behavioral follow-up questions, can be used 
to compare or benchmark organizations within and among 
industries, and can also support longitudinal performance 
studies. 



[0049] The system represents an item by item, group by 
group performance improvement digest that allows manage- 
ment the ability to recognize the linkage between perfor- 
mance of key processes to quality improvement outcomes 
(including the future allocation of resources to achieve such 
outcomes). 

[0050] The system does this by compiling each respon- 
dent's raw data and generating a written digest of an 
organization's strengths and opportunities for improvement. 
Data is gathered and assembled to provide an organization 
with a detailed analysis in the form of a printed report. 
Responses in the database 100 are analyzed as at 140 and 
reported in several ways. The database table is constructed 
to include vertical columns formatted with mathematical 
formulas representing yes or no response totals, raw Likert 
averages, adjusted Likert averages and Organizational Suc- 
cess Ratios ("OSRs"). Dividing the adjusted Likert by the 
raw Likert produces an OSR. This number can be repre- 
sented in the form of a ratio or quotient and generally as a 
two place decimal ranging from 0.01 to a perfect 1,00. The 
OSR is used to profile the system's highest and lowest scores 
in performance improvement, representing current quality 
strengths and weaknesses in the business entity, 

[0051] The OSR data collected from the employees and 
non-employees, if any, is consistent across the organization. 
The results can present a clear profile of strengths and 
current weaknesses from which the business entity can plan 
appropriate actions. Each OSR is then converted to a Bald- 
rige equivalency formula that converts the two place deci- 
mal point to a percentage. The percentage number is then 
applied line by line to re-create the points available in each 
section found in the sample table. The "maximum" points 
for each category of the Malcolm Baldrige National Quality 
Award is listed with the actual score. 

[0052] As illustrated in FIG. 5, a maximum of 450 points 
for the "business results" category may be obtained, even if 
the sample organization returned only 160 points. To convert 
each business category result to a ratio, the total points 
achieved in each category is divided by the overall maxi- 
mum number of points possible for the category. Thus, 160 
divided by 450 equals 0.356 or 35.6 percent. By converting 
each business category measurement to a ratio, the organi- 
zation may easily obtain a quantifiable assessment of its 
performance. 

[0053] It will be seen that the objects set forth above, and 
those made apparent from the foregoing description, are 
efficiently attained. Since certain changes may be made in 
the above construction without departing from the scope of 
the invention, it is intended that all matters contained in the 
foregoing description or shown in the accompanying draw- 
ings shall be interpreted as illustrative and not in a limiting 
sense. 

[0054] It is also to be understood that the following claims 
are intended to cover all of the generic and specific features 
of the invention herein described, and all statements of the 
scope of the invention which, as a matter of language, might 
be said to fall therebetween. 

[0055] Now that the invention has been described, 
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Whai is claimed is: 

1. A method for quantifying predetermined criteria relat- 
ing to the performance of an organization, comprising the 
steps of: 

an array of survey questions for evaluating predetermined 
performance criteria of an organization; 

a survey administrator; 

a data-gathering interface for a predetermined pool of 
respondents to individually record answers to said 
survey questions; 

a database accessible by said survey administrator con- 
taining answers submitted by said predetermined pool 
of respondents; and 

a predetermined formula for evaluating said answers; 

whereby members of said predetermined pool of respon- 
dents individually record answers to said survey ques- 
tions through said data-gathering interface; 

whereby said survey administrator stores said answers in 
said database so that said predetermined formula pro- 
duces a quantified result based on said predetermined 
criteria relating to the performance of said organiza- 
tion. 

2. The method of claim 1, wherein a dual tone multi- 
frequency telephone system is utilized for said data-gather- 
ing interface and wherein individual members of said pre- 
determined pool of respondents dial into said telephone 
system maintained by said survey administrator, said tele- 
phone system audibly plays prerecorded questions based on 
said predetermined criteria, and said individual members are 
prompted to depress a particular key on a standard touch- 
tone telephone in response to said prerecorded questions 
which responses are recorded in said database. 

3. The method of claim 1, wherein a remote web server is 
utilized to record said answers whereby said individual 
members of said predetermined pool of respondents access 
a website server on the Internet, said website containing said 
survey questions and an input means for said answers to be 
posted through said website server to said database. 

4. The method of claim 1, wherein said survey questions 
provide for a Boolean response followed by a quantified 
response to the same survey topic. 

5. The method of claim 1, wherein the Malcolm Baldrige 
National Quality Award Criteria as defined by the United 
States National Institute of Standards and Technology is 
utilized for said predetermined formula. 

6. The method of claim 1, wherein said quantified result 
is resolved by the steps of: 

scoring the survey responses relative to a predetermined 
score based on determination on the importance of each 
performance criteria; 



correlating the survey responses for performance criteria 
categories into distinct arrays of data; 

adding numerical values to said arrays of data represent- 
ing separate performance criteria categories; and 

dividing said numerical values by a maximum theoretical 
value of said performance criteria category. 

7. The method of claim 1, wherein said predetermined 
pool of respondents is an internal pool of respondents 
consisting of employees of the organization. 

8. The method of claim 1, wherein said predetermined 
pool of respondents is an external pool of respondents 
consisting of non-employees of the organization. 

9. A method for quantifying predetermined criteria relat- 
ing to the performance of an organization, comprising the 
steps of: 

providing an array of survey questions for evaluating 
predetermined criteria of an organization; 

providing a survey administrator; 

defining a predetermined pool of respondents to whom 
said survey questions are to be propounded; 

using a remote web server to record answers whereby 
members of said predetermined pool of respondents 
access said web server on the Internet, one or more web 
pages present said survey questions and an input form 
for said answers to be posted through said web server 
to a database accessible by said survey administrator; 
and 

providing a predetermined formula for evaluating said 
answers based on the Malcolm Baldrige Natiorjal Qual- 
ity Award Criteria as defined by the United States 
National Institute of Standards and Technology; 

whereby said members of said predetermined pool of 
respondents record answers to said survey questions 
through said web server and said survey administrator 
stores said answers in said database wherein said 
predetermined formula produces a quantified result 
based on said predetermined criteria relating to the 
performance of said organization. 

10. The method of claim 9, wherein said members of said 
predetermined pool of respondents are employees of the 
organization. 

11. The method of claim 9, wherein said members of said 
predetermined pool of respondents are non-employees of 
said organization. 

* + * * * 
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